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The EFF-NRS Project: 

A Win-Win Proposition 

Dear Colleagues: 

HP 

his issue of the EFF Voice is important for all 
of us concerned about improving the qual- 
* ;< y of adult literacy services. It 
focuses on current research and devel- 
opment efforts to develop an assess- 
ment framework for the Equipped for 
the Future (EFF) Content Standards. 

As in all earlier phases of EFF R&D, this work 
engages practitioners in the field, as well as 
researchers and other technical advisors, as part- 
ners in the development process. In this way, we 
can be sure of three critical effects: that 
the approach to assessment developed 
is based on state-of-the-art knowledge 
about how people learn and how to 
measure learning; that it is useful to 
teachers in planning and implement- 
ing instruction in response to learner 
needs; and that it is meaningful to stu- 
dents themselves in defining what 
needs to be learned in order to meet 
specific everyday responsibilities. 

I am excited about the progress of 
this work and about the commitment 
of our state and program partners in 
Maine, Ohio, Oregon, Tennessee, and 
Washington. We could not be making 
this contribution to the field without 
our partners’ willingness to invest time 
and resources in helping us under- 
stand what performance looks like for 
adult learners at every level along the 
continuum — from novice to expert. 

But what excites me most is our 
partnership with the U.S. Department 
of Education’s Division of Adult Edu- 



cation and Literacy (DAEL). The joint National 
Institute for Literacy (NIFL)-DAEL funding and 
support for this project demonstrates our willing- 
ness to collaborate on issues critical to strengthen- 
ing adult education. And it means that the work we 
are doing can really move the whole field forward. 

As most of you know, NIFL has been involved 
for the past 18 months in coordinating the work of 
the National Literacy Summit 2000. 
The summit’s yearlong, nationwide 
consensus-building process resulted in 
the report “From the Margins to the 
Mainstream: An Action Agenda for 
Literacy,” whose findings strongly reinforce the 
importance of EFF. 

EFF offers essential tools for helping us achieve 

(See Message from the Director, page 2) 



A Message from Ron Pugsley, Director of the DAEL, 
Office of Vocational and Adult Education: 

In response to the Department’s need to track and report 
program outcomes under the Adult Education and Family 
Literacy Act, the Division of Adult Education and Literacy 
has been working closely with practitioners to develop an 
accountability system that will effectively tell our story. 

EFF’s goals are clearly related to this effort. Some of the 
core measures of performance in the accountability system 
reflect mastery of skills that EFF has identified as essential 
content knowledge. Where the Department’s system reports 
learners’ progress in multiple levels of language and basic skill 
development, EFF is developing a performance continuum for 
each of its 1 6 Content Standards and plans to develop 
assessments of learner progress through that continuum. 

It is our hope and expectation that this current joint effort 
will inform the work of defining a continuum of performance 
and provide a set of performance benchmarks to provide a 
framework for the development of assessment tools to 
measure learner progress. 

We are committed to working with NIFL to assure that, 
upon completion of this work to develop a performance 
continuum and appropriate assessment tools, we will be 
able to include EFF results in the National Reporting System 
(NRS) for adult education. 



A Message from 
Andrew Hartman, 
Director, NIFL 






2 

E F F Voice 
Winter 2001 






Equipped 
^ the ^ture 



WINTER 2000 VOL. 2/NO.I 



The EFF Voice is a quarterly publication of 
the EFF National Center. 

We encourage submission of letters, 
articles, and other items of interest from 
the field of adult education. Please direct 
all correspondence, including editorial 
comments, to: 

Donna Curry, Publications Coordinator 

EFF National Center 

5766 Shibles Hall 

University of Maine 

Orono, Maine 04469-5766 

207/581-2402, ext. 13 

donnac@clinic.net 



The EFF Voice concentrates on a 
specific topic for each issue. The next 
issue will focus on the topic: 

Family Literacy 

If you have information on Family 
Literacy that you would like to see 
included, contact Donna Curry at 
the address listed above. 



EFF HOT (Highlights On Teaching Topics) 
is a new quarterly publication of the EFF 
National Center geared toward EFF 
teachers. Each issue concentrates on a 
specific topic of interest, highlighting 
examples of actual practice and offering 
strategies and tools for using EFF in the 
classroom. Contact Donna Curry (see 
above) for more information. 



Designed by Mary Revenig, DesignWorks 



Integrating EFF Content Standards With the 
U.S. DOEs National Reporting System 



T * he National Institute for Liter- 
acy (NIFL), an independent fed- 
eral organization, and the U.S. 
Department of Education’s Division of 
Adult Education and Literacy (DAEL) 
have agreed to jointly fund a project 
intended to create strong linkages 
between the National Reporting System 
(NRS) and the Equipped for the Future 
(EFF) Content Standards. [For more 
information on the NRS, see the box on 
page 3.] The goal of the project is to col- 
lect sufficient performance data on each 
EFF Standard to enable us to: 
a. Identify representative EFF perfor- 
mance tasks that address the knowl- 
edge and skills necessary for transition 
from one NRS level to the next for up 
to ten EFF Standards, 
b. Identify such transition tasks for move- 
ment between all six Adult Basic Edu- 
cation (ABE) levels and all six English 
for Speakers of Other Languages 



What is an EFF Performance Task? 

It is a learning activity that meets learners’ 
purposes and addresses all components 
of an EFF Standard. It informs all steps 
of the instructional planning, teaching, 
and assessment cycle. It addresses a 
real-world adult activity and can be 
analyzed according to the dimensions 
of performance. 



(ESOL) levels on the NRS. 
c. Develop a rich body of standards- 
based performance descriptors for each 
NRS level, which can be used to vali- 
date and enrich the existing body of 
level descriptors. 

Since 1998, programs in over 25 states 
have been using the EFF framework — 
from intake to instruction to assessment. 
Many of these programs are supported by 
state and federal funding that now 
requires them to use the new U.S. Depart- 
ment of Education NRS to report learner 



Message from the Director, 

continued from page 1 

key summit priorities in the area of pro- 
gram quality. It is based on a broad, cus- 
tomer-driven consensus that reflects a 
major Summit goal: instruction should 
focus on real-life skills that make a differ- 
ence in people’s daily lives. And it is 
aimed at providing new tools for assess- 
ing results, in keeping with the Summit’s 
emphasis on the importance of assess- 
ment data to improving instruction and 
providing clear information on results. 
Finally, our current work with DAEL 
enables us to begin another important 
task called for by the Summit — reconcil- 
ing the goals of EFF and NRS. 

Through the development of perfor- 



mance continua for EFF standards, we 
will be able to show how the standards 
relate to the NRS and can demonstrate 
progress toward NRS levels. And when 
the time comes to revise the NRS, our 
joint NIFL-DAEL project will have pro- 
duced a rich set of data that can be used 
to develop an even stronger system of 
reporting and accountability. We are con- 
vinced that students, educators, policy- 
makers, and the public will all benefit 
from the work we will do together. 

The EFF-NRS project is a win-win 
proposition. We look forward to working 
with our colleagues at DAEL — and with 
all of you — to make it a success. A 
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gains by levels. These programs have 
expressed concerns about how to link 
their work with EFF to the NRS. State 
Directors of Adult Education and 
Regional Coordinators in many 
states have also asked for help 
in developing links between 
EFF and the NRS. 

NIFL/EFF agreed to Assessing 
& Learner 

work toward this goal by Needs 

ensuring that, in defining 
a performance continuum for 
each of the EFF Standards, we would do 
everything we could to facilitate direct 
correspondences with the levels proposed 
in the NRS. We chose to focus on the next 
generation of the NRS, since we knew 
that the EFF research had not progressed 
far enough by the July 2000 deadline. 

Background 

In January 1999, NIFL/EFF began work 
toward developing an assessment frame- 
work for the EFF Standards. First, we 
commissioned a series of papers for inter- 
nal use on assessment-related issues that 
would have to be addressed in order to 
assure that programs and states can reli- 
ably and validly assess and report progress 
in relation to the standards. [One of the 
papers, How Instructors Can Support Adult 
Learners through Performance-Based 
Assessments, by Sri Ananda, is available to 
the public. See page 5 for details.] 

In April 1999, NIFL/EFF began work- 



ing with ten field development sites to 
define performance continua for the EFF 
Content Standards. Teachers at these sites 
were asked to use the standards in 
planning and guiding instruc- 
tion and in documenting and 
evaluating learner perfor- 
mance. Teacher reports 
on learner progress 
were then analyzed in 
relation to cognitive sci- 
ence research on expertise to iden- 
tify four dimensions of performance that 
were important to consider in rating how 
well an individual could use an EFF skill 
to carry out real life tasks. These dimen- 
sions of performance are: 

• Structure of knowledge base, 

• Fluency of performance, 

• Independence of performance, and 

• Range of conditions for performance. 
[For more information about these four 
dimensions of performance, see “The 
Four Dimensions” article on page 9.] 

In January 2000, 15 sites in five states 
(highlighted on pages 6-8 of this issue) 
began collecting snapshots of what adult 
learner performance looks like along a 
continuum based on these four dimen- 
sions. In taking this approach to defining 
the performance continuum for each 
standard, our goal was to make sure that 
the continuum we defined for each stan- 
dard reflected the actual range of adult 
(Continued on page 4) 



Planning 



Teaching 



About The NRS 



The NRS is an outcome-based reporting system for federally funded adult 
education programs, as required by Title II of the Workforce Investment Act. 

The goal of the NRS is to improve the public accountability of adult educa- 
tion programs by documenting their ability to meet federal policy guidelines 
and goals. National reporting of results will be based on the aggregation of all 
states’ aggregated data. However, each state’s outcome data will be used to 
assess whether it is meeting its goals and requirements. Each state is also 
expected to look at each program’s aggregated data in order to ensure that 
student outcomes are met. 



In Their Own Words 

Fran Tracy-Mumford, Delaware 
State Adult Education Director: 

“The effort that EFF is putting into 
linking with the NRS will benefit students, 
programs, and the state. The linkage will 
help states document NRS exit levels for 
federal reporting. Programs will be able to 
translate the six EFF learner performance 
levels to NRS levels. This linkage 
becomes practical for students because 
EFF descriptions translate into immediate 
application as a worker, family member, 
or community member. 

“When learners come to our program, 
they don’t talk in NRS terms — they talk in 
EFF terms. With EFF helping to define the 
performance levels, we will be able to 
translate from "learner speak” to “federal 
speak.” That’s why it’s important that the 
performance continuum now being 
developed have benchmarks. Clear defi- 
nitions to accompany those benchmarks, 
rather than merely a standard score on 
an assessment, will help programs 
translate learning to real-life applications.” 

G^> G^> G^> 

Jim Bowling, Ohio State 
Adult Education Director: 

“This partnership between the National 
Institute for Literacy and the U.S. Depart- 
ment of Education holds great promise 
for moving the field forward in a serious 
and significant way. The potential for 
using the results of this work can make a 
major contribution to improving programs 
and services at all levels — local, state, 
and national.” 
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In Their Own Words 

C^> 

Bob Crotzer, Maine State 
Adult Education ABE Coordinator: 

“EFF helps us to tune into what’s really 
important to the learners; takes us away 
from producing cold — but seemingly very 
objective — standardized tests. EFF is 
helping us to look at assessment that is 
more meaningful and helpful to the 
learner as well as the program. 

“Five years from now, with EFF 
assessment strategies in place, I envision 
adult education being a place where 
learners are directors of their own learn- 
ing. Learners will have expanded their 
sense of responsibility for learning, 
accepting the fact that they themselves 
know best what they need to learn. 
Teachers will be the facilitators of that 
process, but they will be the ones who 
accept the fact that they determine 
whether they learn or not." 

Sharlene Walker, Oregon State 
Adult Education Director: 

“EFF surrounds the intent of the law with 
a framework based on research, theory 
and practice. It provides tools and 
strategies for local programs to connect 
purposeful curriculum with better instruc- 
tion and informative assessment. The 
framework is reflected in Oregon’s history 
of a systems approach to adult basic 
education.” 



(Continued from page 3 ) 

learner performance — especially at the 
“novice” end of the continuum which has 
been underrepresented in existing assess- 
ment systems. 

To make sure that teachers were 
focused on the same construct, they used 
standardized templates created by the EFF 
Assessment Team and a standard data col- 
lection reporting protocol to collect data 
on student performance. This protocol 
guided teachers in using the standards 
and dimensions to plan and carry out 
lessons centered on well-structured learn- 
ing tasks and to assess learner perfor- 
mance in relation to those tasks. EFF Field 
Research staff worked with teachers to 
ensure that at least three teachers were 
collecting data on each standard (more in 
the case of the five communication stan- 
dards). These data collection practices 
assured that, for each standard, we col- 
lected data based on the same “construct” 
of the standard that, as a result, can be 
aggregated to build a valid continuum of 
performance for that standard. 

Joint EFF/NRS Project 

The Joint EFF/NRS Data Collection 
Project builds on this past year’s docu- 
mentation, doubling the number of prac- 
titioners collecting performance data in 
these five states. Data collected through 
this project will be used to develop a con- 
tinuum for each standard that represents 
progress in two ways: the tasks get more 
difficult (more complex), and learner 
performance of those tasks becomes 
more proficient. 

In order to build a linkage to the six 
NRS levels for ABE and ESOL, partici- 
pants in the project will work with DAEL, 
State Coordinators, and the EFF Assess- 
ment Team to: 

1. Identify performance tasks for each 
EFF Standard that match the transition 



point from level to level on the NRS. The 

tasks at every level will address all of the 
skill’s components of performance, 
reflecting our assumption that students at 
every level need to be able to use their 
developing skills in daily life. 

2. Use descriptions of student perfor- 
mance in relation to these transition 
tasks to enrich the level descriptors for 
those skills that are currently included 
in the NRS, and to begin developing level 
descriptors for EFF skills that are not yet 
part of the NRS. 

Training and technical assistance pro- 
vided through this project will produce a 
corps of practitioners in all five states 
with considerable expertise in producing 
valid, reliable performance data. This 
should result in better data and report- 
ing overall through the NRS, and enable 
programs using EFF as a framework for 
instruction to report progress using the 
NRS levels. 

The approach to linking EFF Stan- 
dards with NRS levels is based on what 
we now know about the EFF perfor- 
mance continua. As we work with DAEL 
and the five states to collect more data on 
tasks and learner performance over the 
next 12-18 months, we expect to learn 
more about the best ways to measure 
progress and certify learner performance 
on each EFF Standard. At present, our 
intention is to create linkages between the 
two systems, using transition tasks to 
describe what performance looks like as 
learners move from level to level. Our 
goal is to assure that, in the next genera- 
tion of the NRS, adult learning programs 
and states will be able to report learner 
progress from level to level on all 16 EFF 
Standards and to report achievement of 
measurable learner outcomes related to 
the roles of parent/ family member, citi- 
zen/community member and worker. 

Presently, the only achievements of 
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measurable learner outcomes are those 
related to the worker role, and EFF is con- 
tinuing to work with the National Skill 
Standards Board to further develop those. 
However, through this EFF/NRS linkage 
work, we hope to be able to develop mea- 



surable learner outcomes related to the 
other two roles. We hope to develop a 
body of performance descriptors that will 
eventually help programs describe 
achievement in areas related to the family, 
the community, and the workplace. ▲ 



EQUIPPED FOR THE FUTURE ASSESSMENT REPORT: 



How Instructors Can Support Adult Learners 
Through Performance-Based Assessment 



Sri Ananda’s monograph How 
Instructors Can Support Adult 
Learners Through Perfor- 
mance-Based Assessment 
(NIFL, 2000) provides an 
excellent introduction to 
performance-based assess 
ment “as part of a compre- 
hensive system aimed at 
measuring student per- 
formance relative to challenging con- 
tent and performance standards.” 

This handbook provides an orienta- 
tion to the key characteristics of effective 
performance-based assessments. Accord- 
ing to Ananda, these assessments should: 

• demonstrate technical quality, i.e., relia- 
bility and validity; 

• be linked directly to standards; 

• be grounded in theories of learning; 

• serve as both instructional and assess- 
ment tools; 

• be cognitively demanding (requiring 
application and integration of knowl- 
edge); and 

• reflect and accommodate diversity. 
Using performance-based assessments 
that have these key characteristics will 
require a shift in the roles of the instruc- 
tor and the student. 

A performance-based assessment, 
sometimes called alternative or authentic 
assessment, requires a learner to con- 
struct or produce a response to a task or 



stimulus. Rather than 
select a “right” answer 
from a list of choices, 
the learner is required to 
think creatively, reflect 
and evaluate, synthesize, 
and communicate in a 
variety of ways. Under the 
umbrella of performance- 
based assessment are such 
strategies as projects, port- 
folios, performance tasks, computer sim- 
ulations, scientific or mathematical 
inquiries, and research reports. Ananda, 
in her monograph focuses on three 
strategies: written scenarios; projects; 
and portfolios. 

The aim of this handbook is to guide 
practitioners, step by step, through devel- 
oping performance tasks that can be used 
for identification of students’ strengths 
and weaknesses for diagnostic and plan- 
ning purposes, curriculum development, 
and assessment for reporting purposes. 

Presently, the field development pro- 
ject participants have the specific task of 
creating performance tasks, not perfor- 
mance assessments, as part of their 
research and data collection process. 
Eventually, though, performance assess- 
ments will need to be incorporated into 
the adult education assessment system in 
order to assess learners’ ability to apply 
the standards in real-world activities. ▲ 




In Their Own Words 

G^> G^> G^> 

Joan Benz, Instructor at Lane 
Community College and 
Field Assistant for Oregon’s field 
development sites: 

“When we talk about another way to 
identify progress with our learners, teach- 
ers get excited. Up to now, we’ve been 
limited to paper/pencil types of tests and 
teachers tend to teach to those tests 
when that’s what is assessed and 
reported. 

“The biggest change that has hap- 
pened to me because of EFF is that I 
have become much more intentional. I 
now look at what I do and why. I had to 
throw a lot of what I did away — including 
some fun activities. I now have holes in 
my teaching that I have to fix. I like the 
research and solid foundation that EFF is 
based upon. EFF has made me think 
more about pre-assessment — what do 
learners already know and what do they 
still need to know, asking learners what 
vocabulary they already understand — 
again, being intentional in teaching.” 

Donna Miller-Parker, 

ABLE Network (Washington): 

“EFF is consistent with the practices 
and philosophy of adult education here 
in Washington State. Teachers need lots 
of staff development, not because they 
aren’t already doing it, but to help them 
move from the instinctive to the explicit 
level, and to be able to consistently cali- 
brate performance assessment tasks.” 
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A Closer Look at the 
Five States Participating 
in this Years Field 
Development Work 




MAINE 



• Six programs are involved in 
the EFF/NRS field develop- 
ment work. 

• Maine Department of Educa- 
tion has offered regional NRS 
workshops that included 
linkages to EFF to all feder- 
ally-funded ABE programs 
across the state. 



• EFF is built into the state 
plan and is an integral part of 
the delivery system for feder- 
ally-funded ABE programs. 
Financial support for imple- 
mentation is provided to 
individual programs. 

• Each federally-funded ABE 
program has an EFF contact 
person who receives training 
and support from the state’s 
Center for Adult Learning 
and Literacy. These EFF con- 
tact people then provide 
training and support to their 
local district adult education 




staff. A minimum of 12 hours 
of staff development is 
required for ABE teachers in 
local programs. 

• Maine will continue to hold 
both regional and statewide 
sharing sessions so that, as 
an entire state, it can con- 
tinue to develop its programs 
as it implements EFF. 

• The Maine Adult and Com- 
munity Education Team has 
been oriented to EFF and 
supports its implementation. 
All visioning, planning ses- 



sions, and meetings incorpo- 
rate the EFF standards 
framework. One team mem- 
ber is designated as having 
responsibility for assisting 
program administrators with 
the implementation of EFF. 

• A Literacy Council is being 
developed. This Council will 
be responsible for providing 
guidance in the development 
of the vision for Maine Adult 
Education. It will also provide 
support and feedback to the 
state staff on the implemen- 
tation of EFF. 



Equipped 
for the Future 

Timeline 

1994-2004 



Learner 
Response to 
Goal 6 
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Role Maps 
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Four Purposes 
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Identify Key 
Role Activities 
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Role Maps 



1996 



1997 



DEVELOPING THE FRAMEWORK 



1994 
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OHIO 

• Five programs are involved in 
the EFF/NRS field develop- 
ment work. 

• For the third year, Ohio 
has invested in an EFF Pilot 
program. This has been a 
building process from one to 
nine to 30 programs, with 
program support being 
offered by the State Leader- 
ship Network [Ohio Depart- 
ment of Education (ODE), 
Ohio Literacy Resource 
Center (OLRC), four Regional 
ABLE Resource Centers, 

Ohio State University (OSU), 
and Ohio Literacy Network 
(OLN)]. 



• During the last year, Ohio 
offered all ABLE-funded pro- 
grams an opportunity to par- 
ticipate in Program Quality 
Enhancement Grants 
focused on EFF. Thirty pro- 
grams responded, with 
seven programs moving to a 
Phase II level (four programs 
are involved as National Field 
Sites and three programs are 
implementing EFF at a pro- 
gram level), and 27 programs 
at a Phase I, or orientation, 
level. Several programs were 
involved in both levels of 
development. 

• Funds from the Program 
Quality Enhancement Grants 
were used to capture infor- 
mation on how EFF impacts 
different areas of adult edu- 
cation programs, from teach- 
ing to intake to marketing. 

At the end of last year’s grant 
period, documentation 
was collected in The EFF 
Journey. [For more informa- 



tion on The EFF Journey, 
contact Judy Franks at 
330-672-0753.] 

• The OLRC has created 
Eureka!, a database for the 
collection of lesson plans 
developed by the pilots. 
Eureka! is available online 
at http://literacy.kent.edu/ 
eureka. 

• The Ohio Uniform Portfolio 
System (Ohio-UPS) was 
created as an assessment 
and accountability system 
for collecting and analyzing 
student work in order to doc- 
ument student progress and 
advancement from one 
educational level to the next. 
Ways to correlate the EFF 
standards with the Docu- 
mentation Logs are currently 
being explored. 

• Ohio is developing and 
training an 11 -member 
Regional Training Team con- 



sisting of OLRC, ODE, and 
OSU representatives, as well 
as a Regional Resource 
Center staff member and a 
regional program teacher. 
These trainers are building 
their knowledge base and 
will provide regional EFF 
support to the field. 

• A member of the Regional 
Training Team was awarded 
a NIFL fellowship to design 
assessment activities that 
are based on the EFF 
standards but that also 
accommodate Ohio's ESOL 
benchmarks for the six NRS 
levels. Teachers can use 
these activities for documen- 
tation of progress in student 
portfolios. 

• The Regional Resource Cen- 
ter Network will be incorpo- 
rating EFF into core trainings 
and is working together to 
develop an action plan. 
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EFF 
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Review 
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Review 
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Framework that 
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Publish Guide 
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EFF Content 
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Develop Tools 
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DEVELOPING THE STANDARDS 



DEVELOPING THE ASSESSMENT FRAMEWORK 
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• While the language of the 
three roles has been written 
into the state plan, EFF has 
not been officially adopted. 
The State Leadership Net- 
work is working with the EFF 
National Center to develop a 
strategic implementation 
plan with the long-range goal 
of incorporating additional 
phases of EFF development 
into the state plan through 
2004. 




OREGON 

• Five programs are involved in 
the EFF/NRS field develop- 
ment work. 

• The EFF Framework has 
been blended into the state’s 
secondary standards for high 
school completion and GED 
(General Educational Devel- 
opment). These standards 
include performance-based 
assessments. 

• The language of the three 
EFF roles (worker, family 
member, and citizen) has 
been explicitly written into 
the state plan, but EFF has 
not been officially adopted. 

• An EFF Orientation is being 
integrated into the Profes- 
sional Development System 
available to all adult 
educators. 



• Oregon adult educators are 
very receptive to EFF, and 
are requesting more detailed 
training on how to implement 
EFF in their programs. 
Specifically, they’re trying to 
figure out how to implement 
EFF without it being an 
add-on. 

• Oregon practitioners, along 
with the Oregon Department 
of Community Colleges and 
Workforce Development, are 
working on aligning Family 
Literacy, Corrections, Tutor- 
ing, and Adult Basic Skills 
goals to the NRS. 

• Oregon’s philosophy of 
“connectivity” is evidenced 
in its program development 
and improvement model, 
which integrates all the 
different aspects of adult 
education. 




TENNESSEE 

• Three programs are involved 
in the EFF/NRS field devel- 
opment work. 

• Other programs across the 
state are gradually adopting 
EFF one step at a time. For 
example, some programs are 
changing their intake process 
by focusing on learner goals. 
Others may be examining 
one standard and its implica- 
tions in the classroom. 

• Tennessee has incorporated 
EFF into the state plan for 
professional development. 



• EFF was a key criterion in 
the process for awarding 
English Language and Civics 
grants to local programs. 

• The Tennessee Department 
of Human Services has 
funded the revision and 
publication of the Learning 
Skills Curriculum, a four- 
week orientation class based 
on a model developed by the 
Knox County Adult Literacy 
Program, an EFF field site. 
The curriculum, which is built 
around the EFF Standards, 

is being piloted in selected 
programs and will be 
available state-wide next 
year. 

• The Center for Literacy 
Studies (CLS) coordinates 
dissemination of information 
about EFF in the state. Many 
CLS staff participated in field 
site training in October, as 
part of their own professional 
development. 




WASHINGTON 



• Five programs are involved in 
the EFF/NRS field develop- 
ment work. 



• Washington has incorporated 
the EFF Framework into the 
state plan and programs 
receiving ABE funding must 
agree to implement compo- 
nents of the Framework. 



• Washington uses EFF in a 
crosswalk with all state 
agencies that impact adults. 

• Washington’s philosophy of 
adult education has made it 
easier to move all programs 
toward EFF implementation. 

• Washington’s State Literacy 
Resource Center provides 
training and materials related 
to EFF. 

• EFF is linked directly to the 
state’s welfare-to-work 
requirements. 

• EFF has been a requirement 
of Washington’s state-funded 
Families at Work initiative 

for two years. Funding and 
support personnel assisted 
these programs in determin- 
ing how EFF impacts 
instruction. 

• The State Performance 
Task Group is developing an 
interim performance system 
to use over the next four 
years for this generation of 
the NRS. This process 
involves a firm commitment 
by the Department of Adult 
Literacy which continues to 
advocate for practitioner 
input into the development of 
assessment materials that 
include performance-based 
activities. 

• Washington has embraced 
standards for many years, 
including them as part of its 
program monitoring process. 
The challenge has been to 
figure out how to base fund- 
ing on performance, and the 
state is looking to EFF for 
guidance. ▲ 
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The Four Dimensions 
of Performance 



np „ r , . „ 

he Four Dimensions of Performance are being used to inform the 
planning, instruction, and assessment process in the current EFF 
field development sites. Because these four dimensions make sense as 
a way of thinking about student progress, we have seen evidence that teach- 
ers implicitly think about them when they describe what students know and 
can do. However, in order to design effective learning activities that give 
learners an opportunity to show they can use a standard to perform, teach- 
ers and learners need to explicitly consider the four dimensions — knowledge 
base, fluency, independence, and range. 



Let’s take a closer look at an illustra- 
tion of how the four dimensions influ- 
ence how we learn. 

Think about something that you’re 
fairly proficient at. It could be a hobby 
such as golf or gardening or reading. Or 
it could be a common task, like driving 
an automobile. How did you become 
proficient? How do you know you’re 
proficient? 

In answering these questions, you prob- 
ably realized that, as you became more 
proficient, your understanding of what the 
skill or activity required increased. You 



depended less on others — or other 
resources — for help; your ability to per- 
form the activity in different situations 
under increasingly complex conditions 
improved. And, you as you grew more 
confident in your ability to perform the 
activity, some components of it became 
more automatic. 

According to the latest research on 
how people learn, that’s how people move 
from novice to expert. Researchers say 
that the four dimensions of performance 
influence how people move from novice 
to expert in any area . 1 



When you first learned to drive a 
car, you needed to study the rules. 

You had to pass a written driver’s 
test that covered a broad range of 
knowledge to “prove” that you had 
the knowledge to drive safely. Now, 
when you find yourself in particu- 
lar driving situations, you don’t 
think about all those rules — just 
the ones that are related to the par- 
ticular situation. 



1 . Knowledge Base 



Experts not only have a wealth of knowl- 
edge, but they organize their knowledge 
into meaningful patterns around big ideas, 
basic principles, for easy retrieval. They 
have a vast bank of knowledge related to 
their field or subject, and they know when 
and how to readily access the knowledge 
needed at a particular time. According to 
cognitive scientists, experts’ knowledge is 
“conditionalized — it includes a specifica- 
tion of the contexts in which it is useful.” 

( How People Learn, pg. 31) 

(continued on page 1 0) 



How People Learn: Brain, Mind, Experience, and School (John D. Bransford, Ann L. Brown, 
and Rodney R. Cockings, Eds. Washington, D.C.: National Academy Press. 1999). 



In Their Own Words 

Jane Meyer, Director 
Canton, OH ABLE Program: 

“When using the four dimensions of per- 
formance, I saw the teachers thinking 
more about what each individual student 
knew and could do before the lesson and 
using this information to plan instruction. 
Then, looking again at the four dimensions 
of performance after the learning activities 
helped the teachers to report student 
progress in more detail, but much of it was 
teacher observation. We need assessment 
beyond teacher observation to measure 
changes in knowledge base, indepen- 
dence, fluency, and range. Reflecting on 
the four dimensions of performance also 
helped students to better articulate what 
they were learning and why.” 

CV^> 

Eleanor Lockwood, Director 
Windham, ME Adult Education: 

“I am very encouraged by the presence 
of the NRS folks around the EFF table, 
the discussion of retaining six levels, and 
the hope that this work will impact NRS 
four years hence.” 

Kim Gass, Supervisor 
Greenville City Adult Basic 
Education, Greenville, TN: 

“Participation in this research enables 
us to respond to our general public who 
often wonder what we are up to! Our 
administration is noticing that we are not 
the GEDers’ down the street who get the 
leftovers and problems of the community. 
We are a highly accountable, successful, 
and meaningful part of the educational 
community.” 
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In Their Own Words 

C^> 

Regie Stites, Educational Researcher, 
Center for Education and Human 
Services, SRI International 

“The EFF role maps, standards, common 
activities, and — eventually — performance 
continua and benchmarks of perfor- 
mance are part of an integrated 
framework that can help teachers 
align learners’ goals with curriculum 
design and teaching practices and with 
measures of learning progress. Admit- 
tedly, this is a tall order for the teacher, 
and the fact that curriculum and 
assessments used to inform instruction 
are poorly aligned with the standardized 
tests used for external reporting and 
accountability makes the job even harder. 

“To improve on the current ‘misaligned’ 
system, progress needs to be made 
at both the program and systems level. 
Substantial investment in staff develop- 
ment is the starting point for alignment at 
the program level. Staff development 
should provide teachers with the models, 
support, and guided practice they need 
to apply the EFF framework to aligning 
learning goals with teaching practice and 
with measures of learning at the program 
level. At the same time, work is needed 
at the level of accountability systems to 
better capture results that matter.” 



(continued from page 9) 

You probably don’t think twice 
now about the coordination 
needed to start the car, put it in 
reverse, and steer appropriately 
while looking in the rearview mir- 
ror. Think about how challenging 
you thought this was when you first 
began learning to drive — too many 
things to think about all at once! 



Again, think about learning to 
drive. First, you had to have an adult 
in the car with you when you got 
your learner’s permit. As your com- 
fort and knowledge about driving 
improved, you were able to venture 
out alone. 



First, you probably just drove 
around a quiet block. Then you 
drove where there was a bit more 
traffic. As your understanding of 
how other motorists operate and 
your knowledge about the rules of 
the road became more automatic, 
you tried driving on the interstate. 
For at least some of you, the range 
of driving experiences may also 
have included cars without auto- 
matic transmission, or perhaps 
trucks, vans, or even tractors. A 



2. Fluency 



Experts can effortlessly and automatically 
retrieve and apply relevant knowledge in a 
particular situation. This easy retrieval does 
not necessarily suggest that experts can do 
so more quickly than others; in fact, they 
may take more time, because they are capa- 
ble of looking at an issue from more angles 
than those who have less of a knowledge 
base. 



3. Independence 



As individuals move from novice to expert, 
they are able to function with less guidance 
and support as they become more indepen- 
dent in their ability to retrieve and apply 
knowledge to specific situations. Not only 
are experts able to perform with greater 
independence, they are also better able to 
determine what kinds of assistance they 
may need. 



4. Range 



Individuals are able to use skills and knowl- 
edge in increasingly complex tasks as they 
move from novice to expert. The situations 
in which they address tasks becomes more 
varied as their knowledge base increases 
and their performance becomes more fluent 
and independent. 



Equipped for the Future Contact People 



■ National Director: Sondra Stein, 
sstein@nifl.gov, 202-233-2041, 
National Institute for Literacy 

■ Research and Data 
Collection: Brenda Bell, 
bsbell@utkux.utk.edu, 865-974-6654, 
EFF Research Center, Center for 
Literacy Studies, University of 
Tennessee 



■ Staff Development, Technical 
Assistance and Materials: 

Lisa Levinson, LisaL595@aol.com, 
207-581 -2402, EFF National Center, 
University of Maine 

■ Assessment: Peggy McGuire, 
peggymcguire@erols.com, 
215-843-8384, Center for Literacy 
Studies, University of Tennessee 
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For More Information About Equipped for the Future 



On-Line Resources 

■ LINCS is the NIFL’s on-line connection 
to adult literacy information. Region-spe- 
cific collections are available to connect 
state and local organizations, practitioners, 
researchers, and learners. The site 
address is: http://www.nifl.gov/lincs 

■ The Equipped for the Future Website 

is a special collection of LINCS. Visit this 
site to learn more about the EFF initiative. 
Anyone with an active Internet connection 
and a Web-browsing program can down- 
load the EFF Framework and the EFF 
Content Standards document, and can 
find out about current EFF publications, 
announcements, and events. The site 
address is: http://www. nifl.gov/lincs/ 
collections/eff 

■ The Equipped for the Future 
Discussion List is an ongoing, on-line 
conversation about EFF by teachers, 
administrators, and others interested in 
and using EFF. 

To view the archives: 

Go to http://www.nifl.gov/lincs/ 
collections/eff, select “Online Discus- 
sion” from the “Features” menu, and 
choose archives for a specific year. 

To subscribe to the discussion list: 

Go to http://www.nifl.gov/lincs/ 
collections/eff, choose “Subscribe to 
the NIFL-4EFF List” option and follow 
the prompts. (You may also subscribe by 
sending the following message from your 
e-mail account: To: listproc@literacy. 
nifl.gov Subject: leave blank 
Message: Subscribe NIFL-4EFF your 
first name your last name) 

Print Resources Available 

To request single copies of our published 
Equipped for the Future materials, please 
contact the National Institute for Literacy at 
ED Pubs at this toll free number, 1 -800- 
228-8813. For written requests, their 



address is ED Pubs, P.O. Box 1 398, 
Jessup, MD 20794-1398; fax: 301- 
470-1244. The web site is: http://www. 
ed.gov/pubs/edpubs.html, or email 
them at EdPubOrders@aspensys.com 
For multiple copies, please contact the 
Equipped for the Future National Center at 
207-581 -2402, or email Dawn Norris at 
dawn.norris@umit.maine.edu. For all 
publications, please order by the publica- 
tion number in parentheses. 

■ Equipped for the Future: A Cus- 
tomer-Driven Vision for Adult Literacy 
and Lifelong Learning. Published in 

1 995, this book describes the vision 
and research basis of EFF’s initiative for 
reforming the adult literacy and lifelong 
education system. (EX 001 9P) This docu- 
ment can be downloaded at: 
http://www.nifl.gov/ EFF/content.htm 

■ Equipped for the Future: A Reform 
Agenda for Adult Literacy and Lifelong 
Learning. This 1 997 publication builds 
on the previous work, reporting the 
continuing research and development 

of EFF’s content framework for adult 
performance standards. (EX 0020P) 




■ Equipped for the Future Content 
Standards: What Adults Need to Know 
and Be Able to Do in the 21st Century. 

The newly released content standards 
guide describes the EFF development 
process, content standards, how the 



standards have been used in programs 
and where EFF is headed as we develop 
our assessment and reporting systems. 
(EX 0099P) This document can be down- 
loaded at: http://www.nifl.gov/lincs/ 
collections/eff/standards_guide.pdf 

■ Equipped for the Future Research 
Report: Building the Framework, 1 993- 
1997 by Juliet Merrifield. This July 2000 
publication documents the research 
conducted through the summer of 1 997, 
including the process of gathering and 
analyzing data to create the framework, 
and the concepts and theories involved. 
(EX 0106P) This document can be 
downloaded at: http://www.nifl.gov/ 
lincs/collections/eff/merrifield_eff.pdf 

■ Equipped for the Future Assessment 
Report: How Instructors Can Support 
Adult Learners Through Performance- 
Based Assessment. This July 2000 
publication introduces instructors to 
specific performance-based assessment 
methods that show promise for measuring 
progress relative to the EFF Standards. 

(EX 01 10P) This document can be 
downloaded at: 

http://www.nifl.gov/lincs/collections/ 

eff/ananda_eff.pdf 

■ EFF Skills Wheel Posters are still 

available. These 22” x 36” full-color 
posters clearly display the 1 6 EFF skills/ 
standards, along with their relation 
to the four categories of skills. To order, 
please contact: 

Dawn Norris at 207-581-2402 or 
dawn.norris@umit. maine.edu. 

Pricing is as follows: 



1 poster 


$5.00 ea. 


2-4 posters 


$3.00 ea. 


5-9 posters 


$1 .75 ea. 


10-14 posters 


$1 .25 ea. 


15+ posters 


$1-10 63. 
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About the EFF National Center 

The Equipped for the Future National Center at the University of 
Maine was created in 1999 as the staff development, materials, and 
technical assistance center for EFF implementation. The EFF 
National Center’s primary responsibility is to reach out and establish 
strong linkages with key partners, including adult education, family 
literacy, welfare to work, skill standards voluntary partnerships, and 
other workforce development systems, and to assure that these part- 
ners have the support needed to integrate EFF as part of their effort 
to deliver desired results. The activities of the EFF National Center 
include: develop a national network of certified EFF trainers; 
develop materials and products to support EFF adoption and use; 
provide customized training and technical assistance to the key cus- 
tomer systems; and support current EFF users. 

Through its corps of national facilitators and on-going train-the- 
trainer institutes, the Center assures quality EFF staff development. 
If you are interested in discussing how EFF can work within your 
organization, scheduling EFF staff development training, or want to 
discuss EFF implementation in your state, agency, or organization, 
please contact Lisa Levinson, the EFF National Center Director, at 
207-581-2402 ext. 17, or email her at LisaL595@aol.com. A 



Calendar of Events for March 2001 
National Conference 



March 16-20 

Tenth Annual National Conference 
on Family Literacy 

Location: Dallas, TX 
Contact for EFF: Dawn Norris 
Contact for the National Conference: Carmel Mackin, 
Operations Officer - 502-584-1 1 33 
Presenters: Jane Meyer 
Lisa Levinson 
Sondra Stein 

Participants: Family literacy practitioners, administrators, 
policymakers and researchers 



For more information on state and national EFF 
presentations, visit our website: 
http//www.nifl.gov/lincs/collections/eff_news_events 




